While electronic documents stored on computers provide a number of advantages over written documents, many users continue to perform some tasks with printed versions of electronic documents. These tasks include, for example, reading and annotating the documents. With annotations, the paper version of the document assumes particular significance, because the annotations typically are written directly onto the printed document. One of the problems, however, with directly annotating a printed version of a document is the difficulty in later converting the annotations into electronic form. Ideally, electronically stored annotations should correspond with the electronic version of the document in the same way that the handwritten annotations correspond with the printed version of the document.
Storing handwritten annotations in electronic form typically requires a user to review each handwritten annotation and personally enter it into a computer. In some cases, a user may scan the annotations written on a printed document, but this technique creates a new electronic document. The user must then reconcile the original version of the electronic document with the version having the scanned annotations. Further, scanned images frequently cannot be edited. Thus, there may be no way to separate the annotations from the underlying text of the original document. This makes using the annotations difficult.
To address this problem, pens have been developed to capture annotations written onto printed documents. In addition to a marking instrument, this type of pen includes a camera. The camera captures images of the printed document as a user writes annotations with the marking instrument. In order to associate the images with the original electronic document, however, the position of the images relative to the document must be determined. Accordingly, this type of pen often is employed with specialized media having an information pattern printed on the writing surface. The information pattern represents a code that is generated such that the different sections of the pattern occurring around a location on the media will uniquely identify that location. By analyzing or “decoding” this information pattern, a computer receiving an image from the camera can thus determine what portions of the code (and thus what portion of a document printed on the paper) were captured in the image. One example of this type of information pattern is described in U.S. patent application Ser. No. 10/284,412, entitled “Active Embedded Interaction Code,” filed on Oct. 31, 2002, and naming Jian Wang et al. as inventors, which application is incorporated entirely herein by reference. In addition to providing location information, various implementations of this type of information pattern can alternately or additionally be used to represent other types of information as metadata, such as a document identification number.
While the use of such patterned paper or other media allows written annotations on a paper document to be converted into electronic form and properly associated with the electronic version of the document, this technique presents its own difficulties. For example, because the camera is mounted on the pen, both the pen and the writer's hand may affect the quality of the captured images. When writing with a pen, very few users will maintain the pen in a completely vertical direction. Instead, most users will tilt the pen toward their person, toward their person and to their left, or toward their person and to their right. A few users may even tilt the pen away from their person.
The various tilting angle between pen and paper will make the illumination of captured image varies correspondingly. For example, the gray level of an image captured from a blank area will be different from one area to another. Even if the pen includes a light, such as an infrared LED, mounted near the pen tip for illumination, when the pen is tilted the distance between the writing surface and the image sensor will not be uniform, resulting in a non-uniform illumination for the image.
In addition, the printed document itself may obscure areas of the pattern printed on the writing surface of the media. That is, the content making up the document, such as text and pictures, may obscure or occlude portions of the information pattern printed on the writing surface. If the pen captures an image of one of these areas, then the computer may not be able to use distinguish the information pattern from the content. Also, the computer may not accurately recognize the code from the image. For example, if the code is binary, then the computer may erroneously recognize a portion of the pattern representing a “0” value as a “1” value, or vice versa.